1. Field of Invention
The present invention relates generally to imaging systems. More particularly, the present invention relates to constructing an approximation of an image obtained using a single-viewpoint imaging system when the image is actually obtained through the use of a non-single-viewpoint imaging system.
2. Description of the Related Art
Panoramic imaging is often used in imaging applications due to the fact that panoramic imaging systems provide a relatively wide field of view. Although panoramic imaging systems may be used for a variety of different purposes, such systems are generally used in applications which benefit from a wide field of view. Such applications often relate to, but are not limited to, the areas of surveillance, robotics, and machine vision.
In general, panoramic imaging systems include devices which are capable of imaging a wide angle of a scene. The wide angle may be up to approximately 360 degrees as measured as both an azimuth and an elevation. Often, panoramic imaging systems require the use of multiple cameras in order to capture a wide angle of a scene. FIG. 1a is a diagrammatic representation of a panoramic imaging system which uses multiple cameras. Cameras 130 are spaced apart, e.g., camera 130a is spaced apart from camera 130b, such that each camera 130 may effectively capture a portion of a scene 140, which may or may not be reflected off of a mirrored surface.
The use of a multiple camera panoramic imaging system 120 typically enables an image of an entire scene 140 to be captured simultaneously. However, using the approximation that each camera 130 is effectively a pinhole camera, each camera 130 has a different viewpoint. Hence, multiple camera panoramic imaging system 120 is a multiple-viewpoint system. A multiple-viewpoint system, or a system which is not a single-viewpoint system, requires a reconstruction in order to create a single image from the data that is gathered using cameras 130. In other words, data gathered from cameras 130 must be reduced in order to form a single, composite image. Such a reconstruction often results in distortions in the reconstructed image, which is arranged to be projected onto a projection surface for viewing. That is, reconstruction errors are likely to occur. Further, the use of a multiple camera panoramic imaging system may be expensive, and requires a significant amount of physical space, as such a system requires multiple cameras 130.
Another example of a panoramic imaging system is shown in FIG. 1b. As shown, a camera 102 is positioned over a curved mirrored surface 104. Camera 102 is positioned such that it images scenes from mirrored surface 104. By imaging scenes from a mirrored surface, a relatively small field-of-view camera may be used to encompass a much larger field of view. Camera 102 may either be a standard camera, which has a perspective lens, or a specialized camera, which has a telecentric lens. The choice of camera 102 is dependent at least in part upon the curvature of mirrored surface 104. By way of example, when mirrored surface 104 has the curvature of a paraboloid, camera 102 is typically a camera with a telecentric lens. Alternatively, when mirrored surface 104 has the curvature of a hyperbola, camera 102 generally includes a perspective lens.
Mirrored surface 104 may generally be flat, spherically curved, paraboloidally curved, or hyperboloidally curved. While mirrored surfaces which are spherically curved are relatively easy to form, a panoramic imaging system with a spherically curved mirrored surface is a multiple-viewpoint system. On the other hand, while panoramic imaging systems with paraboloidally or hyperboloidally shaped mirrored surfaces are typically characterized by single-viewpoints, such mirrored surfaces are difficult to form. Although mirrors curved as paraboloids and hyperboloids are relatively difficult to manufacture and, hence, expensive, panoramic imaging systems which use such mirrors are single-viewpoint systems. As such, reconstruction of a resultant image, obtained using a single-viewpoint system, that is suitable for projection does not result in a significant distortion.
In a multiple-viewpoint system, optical rays do not converge at a single viewpoint associated with, e.g., within, mirrored surface 104. That is, rays that contact mirrored surface 104 at different elevation angles, i.e., angles at which incoming rays contact mirrored surface 104, do not converge at a single viewpoint within mirrored surface 104. Conversely, for single-viewpoint systems, optical rays which are at any elevation angle all converge to a single viewpoint.
As mentioned above, panoramic imaging systems, such as those which utilize mirror imagery, e.g., camera systems, are used for many different purposes. In some cases, the size of an imaging system is not critical. However, in other cases, the need for a compact imaging system is considered to be important. By way of example, for surveillance purposes and for surgical purposes, compact imaging systems are preferred. For surveillance, a compact imaging system is preferred at least in part to prevent an individual who is being monitored from becoming aware of the fact that an imaging system is recording his or her actions. For surgery such as arthroscopic surgery, a compact imaging system is desired in order to enable the imaging system to be used in small spaces.
In general, minimizing the amount of distortion in a reconstructed image is desirable. That is, the ability to accurately reconstruct images is desirable. For delicate applications such as surgical applications, minimizing the amount of distortion associated with a reconstructed image often proves to be critical. By way of example, when a viewed, reconstructed image is inaccurate, or otherwise has a high level of distortion, a surgeon who is relying on the reconstructed image may make errors in a surgical procedure.
Therefore, what is needed is a relatively inexpensive, compact, accurate panoramic imaging system. More specifically, what is desired is a method for reducing the distortion error associated with reconstructing an image which is obtained using a multiple-viewpoint panoramic imaging system.
The present invention relates to a method and an apparatus for reconstructing images obtained using a multiple-viewpoint imaging system. According to one aspect of the present invention, a method for projecting an image using a panoramic imaging device which includes a first camera includes selecting a single viewpoint that is near a plurality of virtual viewpoints associated with the device. Once the single viewpoint is selected, an optical ray which intersects a surface of the device at a first angle is selected from the plurality of optical rays. The optical ray is then moved or otherwise displaced so that it intersects the single viewpoint while substantially maintaining the first angle. Once the optical ray is displaced, an image is formed using the displaced optical ray to create a reprojection from the single viewpoint. In one embodiment, the panoramic imaging device includes a spherical mirror, and the first camera includes a perspective lens. In such an embodiment, the surface the optical ray intersects is an outer surface of the spherical mirror. By reprojecting an image from a single viewpoint, the distortion associated with a multiple-viewpoint system may be reduced.
According to another aspect of the present invention, an apparatus for imaging a wide field of view includes a camera and a mirrored surface. The camera and the mirrored surface are arranged such that optical rays which intersect the mirrored surface at different angles do not share a single viewpoint. The apparatus also includes a processing system that displaces optical rays to enable the optical rays to intersect the single viewpoint while substantially maintaining the different angles. The processing system also aids in the formation of an image using the displaced optical rays, and reprojects the formed image.
In one embodiment, the processing system is arranged to identify the single viewpoint. In another embodiment, the apparatus also includes a projection surface onto which a formed image may be reprojected. Although the shape of the surface may vary, in such an embodiment, the surface may be either a cylindrically shaped surface or a planar surface.
In accordance with still another aspect of the present invention, a method for mapping an image using a panoramic imaging device that has multiple virtual viewpoints, and includes both a camera and a mirror, involves selecting a single viewpoint. The single viewpoint may be selected to be in proximity to the multiple virtual viewpoints. After the single viewpoint is selected, an incoming optical ray is moved to intersect the single viewpoint such that its elevation angle is maintained. The moved optical ray is than used in the formation of an image. In one embodiment, moving the optical ray includes applying a mathematical algorithm to computationally displace the optical ray.
These and other advantages of the present invention will become apparent upon reading the following detailed descriptions and studying the various figures of the drawings.